A block matching method and a correlation method based on a correlation calculation are known as methods of detecting a motion vector of an image during motion blur correction and the like.
In the block matching method, an input image is divided into a plurality of blocks of an appropriate size (for example, 8 pixels×8 lines). Using these block units, differences in pixel values between a certain region of a current field (frame) and a previous field is calculated. On the basis of this difference, a block of the previous field that has a high correlation with a certain block of the current field is searched for. Thereafter relative displacement between the two blocks is set as the motion vector of the certain block.
In a method of searching for a highly correlated block during block matching, the correlation is evaluated using a sum of squared difference (SSD), which is a sum of squares of the differences in the pixel values, and a sum of absolute difference (SAD), which is an absolute value sum of the pixel value differences. The lower the SSD or SAD value is, the higher the evaluation of correlation is. The SSD and SAD are expressed respectively by following Equations (1) and (2).
                              SSD          ⁡                      (                          I              ,                              I                ′                                      )                          =                              ∑                                          p                ∈                I                            ,                              q                ∈                                  I                  ′                                                              ⁢                                    (                              Lp                -                Lq                            )                        2                                              (        1        )                                          SAD          ⁡                      (                          I              ,                              I                ′                                      )                          =                              ∑                                          p                ∈                I                            ,                              q                ∈                                  I                  ′                                                              ⁢                                                Lp              -              Lq                                                                      (        2        )            
Here, p is a pixel position within a matching reference block region I of the current field, and q is a pixel position (a position corresponding to the pixel position p) within a matching subject block region I′ of the previous field. Accordingly, p and q are quantities taking two-dimensional values. I and I′ respectively denote two-dimensional regions of the current field and the previous field. The term pεI indicates that the coordinate p is included in the region I, while the term qεI′ indicates that the coordinate q is included in the region I′. Pixel values of the pixel positions p and q are indicated by Lp and Lq, respectively.
Meanwhile, in the correlation method based on a correlation calculation, average values Ave (Lp), Ave (Lq) of the pixels pεI and qεI′ included respectively in the reference block region I and the subject block region I′ of the matching operation are calculated. Differences between the pixel values included in the respective blocks and these average values are calculated using a following Equation (3).
                              Lp          ′                =                                                            Lp                -                                  Ave                  ⁡                                      (                    Lp                    )                                                                                                                    1                    n                                    ⁢                                                            ∑                                              p                        ∈                        I                                                              ⁢                                                                  (                                                  Lp                          -                                                      Ave                            ⁡                                                          (                              Lp                              )                                                                                                      )                                            2                                                                                            ⁢                          |                              p                ∈                I                                      ⁢                                                  ⁢                          Lq              ′                                =                                                    Lq                -                                  Ave                  ⁡                                      (                    Lq                    )                                                                                                                    1                    n                                    ⁢                                                            ∑                                              q                        ∈                        I                                                              ⁢                                                                  (                                                  Lq                          -                                                      Ave                            ⁡                                                          (                              Lq                              )                                                                                                      )                                            2                                                                                            ⁢                          |                              q                ∈                                  I                  ′                                                                                        (        3        )            
Next, a normalization cross-correlation NCC is calculated using an Equation (4).NCC=ΣLp′Lq′  (4)
Blocks having a large normalization cross-correlation NCC are determined to be highly correlated blocks, and the displacement between the blocks I′ and I exhibiting the highest correlation is recognized as the motion vector.
When objects or a photography subjects included in an image are stationary, movements within individual regions accords to the movement of the entire image. Therefore, in this case, the block in which the correlation calculation described above is performed may be disposed in an arbitrary fixed position and the motion vector thereof may be calculated.
It should be noted that in certain cases, a highly reliable motion vector cannot be obtained due to the effects of noise or when the block is on a flat portion or adjacent to an edge portion having a larger structure than the block. To eliminate these cases, JP8-163573A and JP3164121B, for example, disclose techniques for performing a reliability determination when calculating a motion vector.
Further, when the object or photography subject included in the image includes a plurality of movements, the motion vector of the entire image may be calculated with the aim of correcting blur, for example. In JP8-251474A, the object is divided into a plurality of regions and an important region is selected from the plurality of regions in accordance with the magnitude of the motion vector, the size of the region, and so on. The motion vector of the selected region is then recognized as the motion vector of the entire image.
In this case, region selecting means select any one of the following important regions (i) to (iv).    (i) The region having the greatest range of the plurality of regions.    (ii) The region having the smallest motion vector of the plurality of regions.    (iii) The region of the plurality of regions that has the greatest range of overlap with a previously selected region.    (iv) One of the region having the greatest range, the region having the smallest motion vector, and the region having the greatest range of overlap with the previously selected region.
Incidentally, when registration such as electronic motion correction is performed on an image pickup apparatus having a lens that has a large aberration, the amount of displacement caused by the aberration in the image varies in each position of the image. For example, in a wide angle lens or the like, the displacement amount caused by image distortion varies at each image height. It is therefore difficult to determine an inter-frame motion vector (a correction vector) between a plurality of images photographed using a lens having a large aberration. The reason for this is that a correspondence relationship between a motion vector determined on an image having no aberration and a motion vector determined on an image having an aberration varies in each image position (at each image height, for example).
For example, when barrel distortion such as that shown in FIG. 1A exists, the inter-frame motion vector is estimated to be smaller than the inter-frame motion vector when no aberration exists in a region having a great image height. Conversely, when pincushion distortion such as that shown in FIG. 1B exists, the inter-frame motion vector is estimated to be larger than the inter-frame motion vector when no aberration exists in a region having a great image height.
JP2007-129587A proposes a technique for dealing with distortion in which electronic blur correction through determination of an inter-frame motion vector is performed on an image that has been subjected to distortion correction.